The present invention relates to a method for measuring the amount of movement of video signals, like television signals. The present invention uses the gradient method which has the advantage of effective hardware implementation, and the present invention uses that gradient method repetitively. The present invention is applicable to a high efficiency coding system of video signals which compensates picture movement between frames, and/or a frame rate conversion in a television system.
A conventional frame rate conversion system of a television signal between NTSC and PAL, or SECAM has the disadvantage that the converted picture blurs when the picture moves, because the conversion is carried out through interpolation between two succesive frames. The quality of a converted picture would be improved by using interpolation which counts for the displacement of a picture.
The picture coding which compensates a picture movement has been recently developed and for that purpose, the measurement of the amount of movement of a picture is requested.
Some of conventional measurement systems for that purpose are; (1) Pourier transform method; (2) A method which provides the amount of movement by the displacement which provides the maximum mutual correlation coefficients; (3) A matching method which provides the amount of movement by the displacement which provides the minimum difference between frames or fields; and (4) A gradient method which provides the displacement based on the relation between the spatial gradient of intensity of a picture and the difference between frames or fields.
Among them, the first three methods have the disadvantage that a lot of calculation is necessary and the hardware implementation is difficult. The fourth method can be implemented easily in hardware but it has the disadvantage in the accuracy of the measurement obtained.
A prior gradient method is shown in "Motion Compensated Television Coding" Part 1, in The Bell System Technical Journal, vol 58, No. 3, March 1979, pages 631-635, and in "Estimating the Velocity of Moving Images in Television Signals" in Computer Graphics and Image Processing (1975) 4, pages 311-327.
The prior gradient method discussed in the above citations is described in detail below.
First, a simple linear movement of one dimension is described in accordance with FIG. 1, in which the horizontal axis shows a position, and the vertical axis shows an intensity. It is assumed that the intensity of each picture element in a preceding frame is shown in the curve (a), and the intensity of the same in the present frame is shown in the curve (b). In that case, the symbol (d) shows the frame difference which is the difference of intensity between frames, the angle (A) shows the gradient of the intensity, and the symbol (v) shows the displacement of a picture between frames. It should be noted in FIG. 1 that the displacement (v) is expressed by v=-d/(tan A). Accordingly, the displacement or the amount of movement is obtained by measuring an intensity difference between frames, and an intensity gradient.
Next, a displacement of two-dimension.
It is assumed that a picture I(x,y) moves by the movement vector .alpha.=(a,b) in one frame period, and becomes I(x-a, y-b). When .alpha. is small, the Taylor series of the primary term is shown below. EQU a(.differential.I/.differential.x)+b(.differential.I/.differential.y)=-d(x, y) (1)
where d(x,y) is difference between frames. The equation (1) is expressed as follows. EQU .alpha..multidot.grad I=-d(x,y) (2)
To solve the equation (2) for the components (a) and (b) of the movement vector (.alpha.) under the minimum square-error condition, the followings are obtained. EQU a=-.SIGMA..DELTA.X.DELTA.T/.SIGMA..DELTA.X.sup.2 ( 3) EQU b=-.SIGMA..DELTA.Y.DELTA.T/.SIGMA..DELTA.Y.sup.2 ( 4)
where .DELTA.X, .DELTA.Y, and .DELTA.T represents .differential.I/.differential.x, .differential.I/.differential.y, and d(x,y), respectively, and the following equation which is good for most pictures is assumed. EQU .SIGMA..DELTA. EQU X.DELTA.Y=0 (5)
The equations (3) and (4) are approximately as follows. EQU a=-.SIGMA..DELTA.T sign(.DELTA.X)/.SIGMA..vertline..DELTA.X.vertline.(6) EQU b=-.SIGMA..DELTA.T sign (.DELTA.Y)/.SIGMA..vertline..DELTA.Y.vertline.(7)
The values (a) and (b) are outputs of the measured movement. In the equations (6) and (7), .DELTA.T is the difference between frames, .DELTA.X is the horizontal intensity gradient, .DELTA.Y is the vertical intensity gradient, .SIGMA..vertline..DELTA.X.vertline. is the sum of the absolute of the horizontal intensity gradient of all the picture elements in a block, .SIGMA..vertline..DELTA.Y.vertline. is the sum of the absolute of the vertical intensity gradient of all the picture elements in a block.
The conventional gradient method measures the displacement merely by addition, sign calculation, and a single division according to the equations (6) and (7), therefore, that gradient method can be implemented easily by conventional hardware.
However, the prior gradient method has the disadvantage that the accuracy of the measurement is poor, because of the presumption of the equation (1) which presumes that the movement vector (.alpha.) is small, the presumption of the equation (5), and the approximation of the equations (6) and (7). In particular, the accuracy is poor when the movement is large. The accuracy of the prior gradient method depends upon the size of a picture block used for measurement. When the block is large, the accuracy is improved because characteristic parts in a picture can easily be recognized, and the relative amount of movement as compared with the size of the block is small. On the contrary, when the block is small, the accuracy is poor.
Accordingly, in the conventional gradient method, in order to improve the accuracy for a large movement, an extremely large block must be used. However, the use of such large blocks results in the disadvantages that a fine movement can not be measured, and a large number of calculations are necessary.